A Neural Network model with Bidirectional Whitening

We present here a new model and algorithm which performs an efficient Natural gradient descent for Multilayer Perceptrons. Natural gradient descent was originally proposed from a point of view of information geometry, and it performs the steepest descent updates on manifolds in a Riemannian space. In particular, we extend an approach taken by the "Whitened neural networks" model. We make the whitening process not only in feed-forward direction as in the original model, but also in the back-propagation phase. Its efficacy is shown by an application of this "Bidirectional whitened neural networks" model to a handwritten character recognition data (MNIST data).Comment: 16page

Laplace's rule of succession in information geometry

Laplace's "add-one" rule of succession modifies the observed frequencies in a sequence of heads and tails by adding one to the observed counts. This improves prediction by avoiding zero probabilities and corresponds to a uniform Bayesian prior on the parameter. The canonical Jeffreys prior corresponds to the "add-one-half" rule. We prove that, for exponential families of distributions, such Bayesian predictors can be approximated by taking the average of the maximum likelihood predictor and the \emph{sequential normalized maximum likelihood} predictor from information theory. Thus in this case it is possible to approximate Bayesian predictors without the cost of integrating or sampling in parameter space

Information geometry of density matrices and state estimation

Given a pure state vector |x> and a density matrix rho, the function p(x|rho)= defines a probability density on the space of pure states parameterised by density matrices. The associated Fisher-Rao information measure is used to define a unitary invariant Riemannian metric on the space of density matrices. An alternative derivation of the metric, based on square-root density matrices and trace norms, is provided. This is applied to the problem of quantum-state estimation. In the simplest case of unitary parameter estimation, new higher-order corrections to the uncertainty relations, applicable to general mixed states, are derived.Comment: published versio

Complexity Measures from Interaction Structures

We evaluate new complexity measures on the symbolic dynamics of coupled tent maps and cellular automata. These measures quantify complexity in terms of $k$-th order statistical dependencies that cannot be reduced to interactions between $k-1$ units. We demonstrate that these measures are able to identify complex dynamical regimes.Comment: 11 pages, figures improved, minor changes to the tex

The volume of Gaussian states by information geometry

We formulate the problem of determining the volume of the set of Gaussian physical states in the framework of information geometry. That is, by considering phase space probability distributions parametrized by the covariances and supplying this resulting statistical manifold with the Fisher-Rao metric. We then evaluate the volume of classical, quantum and quantum entangled states for two-mode systems showing chains of strict inclusion

From neurons to epidemics: How trophic coherence affects spreading processes

Trophic coherence, a measure of the extent to which the nodes of a directed network are organised in levels, has recently been shown to be closely related to many structural and dynamical aspects of complex systems, including graph eigenspectra, the prevalence or absence of feed-back cycles, and linear stability. Furthermore, non-trivial trophic structures have been observed in networks of neurons, species, genes, metabolites, cellular signalling, concatenated words, P2P users, and world trade. Here we consider two simple yet apparently quite different dynamical models -- one a Susceptible-Infected-Susceptible (SIS) epidemic model adapted to include complex contagion, the other an Amari-Hopfield neural network -- and show that in both cases the related spreading processes are modulated in similar ways by the trophic coherence of the underlying networks. To do this, we propose a network assembly model which can generate structures with tunable trophic coherence, limiting in either perfectly stratified networks or random graphs. We find that trophic coherence can exert a qualitative change in spreading behaviour, determining whether a pulse of activity will percolate through the entire network or remain confined to a subset of nodes, and whether such activity will quickly die out or endure indefinitely. These results could be important for our understanding of phenomena such as epidemics, rumours, shocks to ecosystems, neuronal avalanches, and many other spreading processes

Geometric derivation of the quantum speed limit

The Mandelstam-Tamm and Margolus-Levitin inequalities play an important role in the study of quantum mechanical processes in Nature, since they provide general limits on the speed of dynamical evolution. However, to date there has been only one derivation of the Margolus-Levitin inequality. In this paper, alternative geometric derivations for both inequalities are obtained from the statistical distance between quantum states. The inequalities are shown to hold for unitary evolution of pure and mixed states, and a counterexample to the inequalities is given for evolution described by completely positive trace-preserving maps. The counterexample shows that there is no quantum speed limit for non-unitary evolution.Comment: 8 pages, 1 figure

An Information-Geometric Reconstruction of Quantum Theory, I: The Abstract Quantum Formalism

In this paper and a companion paper, we show how the framework of information geometry, a geometry of discrete probability distributions, can form the basis of a derivation of the quantum formalism. The derivation rests upon a few elementary features of quantum phenomena, such as the statistical nature of measurements, complementarity, and global gauge invariance. It is shown that these features can be traced to experimental observations characteristic of quantum phenomena and to general theoretical principles, and thus can reasonably be taken as a starting point of the derivation. When appropriately formulated within an information geometric framework, these features lead to (i) the abstract quantum formalism for finite-dimensional quantum systems, (ii) the result of Wigner's theorem, and (iii) the fundamental correspondence rules of quantum theory, such as the canonical commutation relationships. The formalism also comes naturally equipped with a metric (and associated measure) over the space of pure states which is unitarily- and anti-unitarily invariant. The derivation suggests that the information geometric framework is directly or indirectly responsible for many of the central structural features of the quantum formalism, such as the importance of square-roots of probability and the occurrence of sinusoidal functions of phases in a pure quantum state. Global gauge invariance is seen to play a crucial role in the emergence of the formalism in its complex form.Comment: 26 page

Generalized geometric quantum speed limits

The attempt to gain a theoretical understanding of the concept of time in quantum mechanics has triggered significant progress towards the search for faster and more efficient quantum technologies. One of such advances consists in the interpretation of the time-energy uncertainty relations as lower bounds for the minimal evolution time between two distinguishable states of a quantum system, also known as quantum speed limits. We investigate how the nonuniqueness of a bona fide measure of distinguishability defined on the quantum-state space affects the quantum speed limits and can be exploited in order to derive improved bounds. Specifically, we establish an infinite family of quantum speed limits valid for unitary and nonunitary evolutions, based on an elegant information geometric formalism. Our work unifies and generalizes existing results on quantum speed limits and provides instances of novel bounds that are tighter than any established one based on the conventional quantum Fisher information. We illustrate our findings with relevant examples, demonstrating the importance of choosing different information metrics for open system dynamics, as well as clarifying the roles of classical populations versus quantum coherences, in the determination and saturation of the speed limits. Our results can find applications in the optimization and control of quantum technologies such as quantum computation and metrology, and might provide new insights in fundamental investigations of quantum thermodynamics

A Bivariate Measure of Redundant Information

We define a measure of redundant information based on projections in the space of probability distributions. Redundant information between random variables is information that is shared between those variables. But in contrast to mutual information, redundant information denotes information that is shared about the outcome of a third variable. Formalizing this concept, and being able to measure it, is required for the non-negative decomposition of mutual information into redundant and synergistic information. Previous attempts to formalize redundant or synergistic information struggle to capture some desired properties. We introduce a new formalism for redundant information and prove that it satisfies all the properties necessary outlined in earlier work, as well as an additional criterion that we propose to be necessary to capture redundancy. We also demonstrate the behaviour of this new measure for several examples, compare it to previous measures and apply it to the decomposition of transfer entropy.Comment: 16 pages, 15 figures, 1 table, added citation to Griffith et al 2012, Maurer et al 199